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NON-ZERO-SUM STOPPING GAMES IN DISCRETE TIME 


ZHOU ZHOU 


Abstract. We consider two-player non-zero-sum stopping games in discrete time. Unlike Dynkin 
games, in our games the payoff of each player is revealed after both players stop. Moreover, each 
player can adjust her own stopping strategy according to the other player’s action. In the first part 
of the paper, we consider the game where players act simultaneously at each stage. We show that 
there exists a Nash equilibrium in mixed stopping strategies. In the second part, we assume that 
one player has to act first at each stage. In this case, we show the existence of a Nash equilibrium 
in pure stopping strategies. 


1. Introduction 

As a classical model of stopping games, Dynkin game has attracted a lot of research. We refer 
to [3ltl7j and the references therein. In a Dynkin game, each player chooses a stopping strategy, 
and the payoffs are revealed when one player stops. In other words, the game ends at the minimum 
of the stopping strategies. In practice, it is more often that, even if a player has made the decision 
first, her payoff can still be affected by other players’ decisions later on. Therefore, it is more 
reasonable to let the game end at the maximum of the stopping strategies. Moreover, a wise player 
would adjust her strategy after she observes other players’ actions. Based on these two points, 
recently mmm study the stopping games with these features. In particular, mm consider the 
zero-sum case, and [T] investigates the non-zero-sum case in continuous time. 

In this paper, given a filtered probability space (D, T, ,T, P); we consider a non-zero-sum 

stopping game in discrete time 

u^{p,t)=E[U\p,t)], i = l,2, 

where the first (resp. second) player chooses p (resp. r) to maximize the payoff v} (resp. v?). Here 
W{s,t) is JTjvrineasurable instead of J^sAt-measurable as is assumed in Dynkin games. That is, 
the game ends at the maximum of p and r. Moreover, here p and r are not (randomized) stopping 
times, they are strategies that can be adjusted according to each other. 

In the first part of the paper, we consider the case when the two players act simultaneously at 
each stage (here both “stop” and “not stop” are actions). We show that there exists a perfect 
Nash equilibrium in mixed stopping strategies. The main idea to prove the result is to convert the 
original problem to a non-zero-sum Dynkin game with randomized stopping times. 

In the second part of the paper, we consider the game where one player acts first at each stage. In 
this case, we show that there always exists a perfect Nash equilibrium in pure stopping strategies. 
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We use the idea in [18] to prove this result. That is, we first construct saddle points for some related 
zero-sum stopping games, and then using these saddle points we construct a Nash equilibrium for 
the non-zero-sum games. 

This paper extends the result in [T8| to the discrete time case. It has a broad range of applications, 
e.g., when companies choose times to enter the market, or when investors who both long and short 
American options choose times to exercise the options. 

The paper is organized as follows. In the next section, we consider the non-zero-sum stopping 
game when two players act simultaneously at each stage. In Section 3, we study the case when 
one player acts first at each stage. In Section 4, we make a comparison between our discrete-time 
results in this paper and the continuous-time result in [T8] . 

2. Stopping games where players agt simultaneously at each stage 

In this section, we consider the non-zero-sum stopping game where players act simultaneously 
at each stage. We will consider mixed stopping strategies for the game. Theorem 12.51 is the main 
result of this section. 

Let (n, (Tt)t=o,... ,T, P) be a filtered probability space, where 12 is countably generated, and 

r G N is the finite time horizon. Let T be the set of stopping times taking values in {0,... ,T}. 
For any a ^T, let Ta ■= {p €T, p > r}, and %+ := {p G T, p > {t + 1) AT}. Define 

T“ := {0 : {0,... , T} X D i-A {0,... , T} : •) G %+, t = 0,... ,T}. 

Let be the set of randomized stopping times. That is, for any a G T^, a : [0,1] x D i-A {0,... ,T} 
is 13{[0, 1]) 0 T'-measurable, and a{p, ■) € T for any p G [0,1]. 

Definition 2.1. {po,pi) is said to be a (pure) stopping strategy of type A, if p^^T and pi G T“. 
Denote T“ as the set of (pure) stopping strategies of type A. 

For {pq,pi) G T“, pq represents a player’s initial (pure) stopping strategy, and pi{t,-) represents 
the strategy adjusted by the player after she observes the other player’s stopping at time t. For 
p = {pQ,pi),T = (ro,Ti) G denote 

P^\ ■ Pol{po<To} A' Pl(To)l{pg>ro}- 

Definition 2.2. (a, pi) is said to be a mixed stopping strategy of type A, if a & and pi G T“. 
Denote as the set of mixed stopping strategies of type A. 

Remark 2.3. One can also randomize the strategies in T*. However, it turns out that we only need 
to randomize players’ initial stopping times (i.e., the first component of {po,pi) G in order to 

get the existence of a Nash equilibrium for the stopping game introduced below. 

For i = 1,2, let W : {0,... ,r} x {0,... ,r} x D !->■ M, such that W{s,t, •) is T'^vt-iPieasurable. 
For simplicity, we assume that I/® is bounded for i = 1,2. Consider the non-zero-sum stopping 
game 

u\p,t) = [ r {p{p,-),T{q,-))dpdq, /9,rGT“®’, z = l,2, 

4[o,i]2 
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where for i = 1, 2 and ( = (Co, Ci); ^ = ivo, fli) ^ 

r(C,r/) = K[U\C[r^],v[C])] 

= E [f/*(Co;??l(Co))l{Co<r)o} + ^\CliVo),Vo)'^{(o>Vo} + (Co, Co)lCo=r)o}] ' 

Here the hrst player chooses p to maximize and the second player chooses r to maximize v?. 
Recall the definition of a Nash equilibrium. 

Definition 2.4. (p*,r*) € is said to be a Nash equilibrium in for the game ()2.1I) . if for 

any p,T ^ 

u^{p,T*) < u^{p*,T*) and u^{p*, t) <u^{p*, t*). 

Below is the main result of this section. 

Theorem 2.5. There exists a Nash equilibrium in for the game m- 

Remark 2.6. We cannot guarantee the existence of a Nash equilibrium for the game mi if 
players only use pure stopping strategies of type A (i.e., T“j. Consider the following deterministic 
one-period example. Let T = 1 and u^{s,t) = —u‘^{s,t) = l{s=H for s,t = 0,1. Then it is easy to 
see that = {0,1}. Obviously there is no Nash equilibrium for (12.11) in T“. 

Proof of Theorem, \ 2., 51 For t = 0,... , T, let 

F/:= esssupEt[C/^((T, t)] and := esssupEt[17^(t, fj)], 
cr&Tt+ cr&Tt+ 

where E6i[-] := E[-|F6)] for 6 ^T. For f = 0,... , T, let p\{t, •) G %+ and r*(t, •) € Tt+ be optimizers 
for and respectively. That is, 

MU\p*i{t)N)]=Yt^ and ¥.t[U\LTl{t))]=Xl a.s.. 

Obviously p\{-, •) G T“. For t = 0,... , T, define 

Xl :=Et[U\t,Tntm ■.= Et[U\pUt),t)], and Zl = U\t,t), i = 1 , 2 . 

Now consider the non-zero-sum Dynkin game with randomized stopping times 

u\a,f5)= f (E [Xfl^a<i 3 }+Ypl^a>p} + Yfl{a=p}]) dpdq, i = 1,2, (2.2) 

J[o,W 

for a, fi ^ T^'. By [5l Theorem 2.1], there exists a Nash equilibrium {a*, 13*) G (T’’)^ for the Dynkin 
game (12.21) . That is, for any a,/3 € T'', 


u^{a,j3*)<u^{a*,f3*) and vf {a*, (3) < {a*, jd*). 


(2.3) 
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Let := {a*, pI) and := (/3*,rf). Now let us show that {Pm^'^m) ^ is a Nash 

equilibrium for the game (12.ip . Take p = {a,pi) E We have that 


< 


< 


'[ 0 , 1 ]= 






(E [E„[17na,rr(a))]l{„<^*} +E;3*[C/'(/Oi(r),r)]l{a>/3*} + ^'(«>«)l{a=/3*}]) 
(E dpdq 

(e *} + ^/3^*l{a*>/3*} + Z^*'^{a*=p*}]) dpdq 


= / (E + U\plin,nha*>p*} + U\a*,a*)l{a^=p*}]) dpdq 

■1[0,1] = 

where we use (12.3p for the fourth (in)equality. Similarly, we can show that for any r E T“'’, 

u^{p*m,r) < U^{p*^,T^). 

This completes the proof of the result. □ 


3. Stopping games where one player acts first at each stage 

In this section, we consider the stopping games in which one player acts first at each stage. We 
show that there always exists a Nash equilibrium in pure stopping strategies. Theorem 13.31 is the 
main result of this section. 

Let 

T'-:= {lA : {0,... ,r}xL!^{0,... ,T} : ■)€%}. 

Here ip £ represents a player’s (player 2) strategy adjusted at the time when the other player 
(player 1) stops. In other words, player 1 acts first at each stage. (Compare T'* with T“.) 

Definition 3.1. is said to be a (pure) stopping strategy of type B, if tq £ T and ri E T'’. 

Denote T'’ as the set of (pure) stopping strategies of type B. 

For any p = {po,pi) E T“, r = (tq, n) E T^ 

p{t) ■ PoIipo^to} T Pi(T))1{po>to} and t{p) . 1'o1{to<po} “k ti(po)1{to>po}' 

Consider the non-zero-sum stopping game 

w^{p,t) :=E[U\p{t),t{p))] = E [C/*(po,Ti(po))l{po<ro} + 'ro)l{po>ro}] , (3-1) 

for p = {po,pi) £ T“, t = (ro, n) E T'’ and f = 1,2. 

Definition 3.2. {p*,T*) E T“ x T'’ is said to be a Nash equilibrium for the game (13.11) . if for any 
p E T“ and r E T'’, 


w^{p,T*) < w^{p*,T*) and w^{p*, t) < w^{p*, t*). 
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Below is the main result of this section. 

Theorem 3.3. There exists a Nash equilibrium for the game (13.ip . 

We will use the idea in [18] to prove Theorem 13.31 To be more specihc, we will use the saddle 
points of some zero-sum stopping games to construct a Nash equilibrium for the non-zero-sum 
game dHU). We will first provide some results in the zero-sum case in Section 3.1. Then we prove 
Theorem 13.31 in Section 3.2. 

3.1. Zero-sum case. We consider the stopping game in the zero-sum case, i.e., when = —17^ = 
U. We will construct a saddle point for the zero-sum game. (The results in this section are essentially 
provided in |2|. We present them for the completeness of this paper.) 

For any u € T, consider the zero-sum stopping game 

:= esssupessinfEo-[17(/9(T),r(/9))] , (3-2) 

and 

Va := ess inf esssupEo-[[/(/3(r), r(/3))] , (3-3) 

where 

:= {(/0o,/5i) G: Po>(y} and := {(po,/^i) €‘3^'’: Po > cr}- 
For t = 0,... , T, let 


Ft := essinf Et[[/(t, ^)] and Gt := esssupEt[[/(^,f)] V F), 

^671 y ^e7i+ J 

and p(t, •) G Tt+ and r(t, •) G T be optimizers for esssup^g-j^_|^ Et[17(^, t)] and Ft respectively. Since 
F < G, we have that 

Va := ess sup ess inf Eo- [-^pl{p<T} + G'rl{p>T}] = ess inf ess sup E^- [Fpl{p<r} + G'Tl{p>r}] > (3-4) 

peTa tgT^ pfz-ji 

and {pa,Ta) is a saddle point for the Dynkin game (j3.4p . where 

Pa := inf{t > a : Vt = Ft} and := inf{t > a : Vt = Gt}- 
That is, for any p,T € Ta, 

E(t [^P^{p<Ta} + ^'ra^{p>Ta}\ — \_^P<T^ {pa<Ta} ^Ta^{pa>Ta}\ — \j^Pa^{pa<T} + ^t1{p^>t}] • 

Let p* := (Pct,p) G T“ and r* := (ro-,f) € 

Proposition 3.4. We have Va = = v^- Moreover, {p*^,Tf) is a saddle point of the game (13.21) 

and (|3.3p . 
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Proof. Take r = (ro,Ti) € T^. We have that 

^<7[Uip*{T),T{pl))] = [UiPa,Tl{pa))l{p^<ro} + U{p{To),To)l^p^>ro}] 

= E<7 [Ep,[C/(/9,7,'ri(P<7))]l{p,<ro} +Ero[C^(p(To),ro)]l{p^>^o}] 

— [-^Po-l{pCT<ro} + ^Tol{pa->To}] 

> Eo- [-Ep^ p GTa^{p„>T^}\ 

= E„ [C/(/3a,r(/9^))l{p^<^4 + 

= ¥.^[U{pI{t:)X{pI))]. 

where for the third and sixth (in)equalities we use the fact that, on {t < po-}, Gt > vt > Ft, and 
thus Gt = esssup^g 7 '^_^ Et[C/(,f,t)] = Et[U{p{t),t)]. 

Similarly, we can show that for any p G T“, 


E. [u{p{t:),t:{p))] < E. [u{p:{t:),t:{pI))] . 

This completes the proof of the result. 


□ 


3.2. Proof of Theorem 13.31 We will use the saddle points of some related zero-sum stopping 
games to construct a Nash equilibrium for the non-zero-sum game m- 
For t = 0,... , T, let 

F/ := essinfEt[C/^(f,,f)] and := ( esssupEt[C/^(,f,t)] ) VF/, 

y ^ert+ J 

Fj^ := esssupEj[C/^(t,^)] and := ( essinf Ej[[/^(^,t)] ) A F^^, 

eeTt V Sert+ J 

and 

vl := ess inf ess sup E* [F/ -k GJ = ess sup ess inf E* [F^^ l{p<r} + G't l{p>r}] , (3.5) 

"^674 p^Tt p&Tt ’■Gu 

Vt := ess sup ess inf Et [F4^1{p<r} + G'?l{p>r}] = ess inf ess sup E* [Ft^l{p<r} + G'?l{p>r}] • (3.6) 

rSTt PG7t peTt r&Tt 

Let f^(t), p^(t), f^(t), p^(t) be optimizers for F/, ess sup^g- 7 ^_|^ Et[G^(^, t)], F^^, essinfggT^^ Et[G^(^, t)] 

respectively. Let 

Hi :=Et[U\t,f\t)] and :=Et[U\p\t),t)] 

for t = 0,... , T, and 

:= inf{t > 0 : vj < H}} and := inf{f > 0 : < Hf A F^}, 


and 


Define 


p ^2 := inf{t > p2 : vf = F^} and t\ := inf{f > p^ : = G}}. 


p\ lfp^<p^ \ P {t), if t > p^-k 1 and p^ > p^ 

Po=\ , , , Pi{t) = S , 

pG, if p > p , P (i), otherwise, 


(3.7) 
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^0 — 

■ At" 


t},u 

if ij} > /i^, 


rm = 


f^{t), if f > /i^ + 1 and jj,^ < , 

T^{t), otherwise, 


for f = 0,... , T, and 


P*-=iPo^Pi) and T* := (ro*,ri*). 


It can be shown that p* G T“ and r* € T^. 

Proposition 3.5. {p*,T*) is a Nash equilibrium for the game (13.Ih . Therefore, Theorem \S. ,91 holds. 
Proof. Part 1: We will show that 

w\p,T*)<w\p*,T*) (3.8) 

for any p € T“. As , 

p^ < pI := inf{t > 0 : v} = F}}. (3.9) 

Hence, on {t < p^} we have that G} > > Ff, and thus Gj = esssup^g7-^^Et[17^(^,t)] = 

Et[U\p\t),t)]. Then 

w\p*,T*) = E[u\p\f\p^))i{^,^^.y + u\p\p^),p^)i{^i^^.y] 

Now take p = {pQ,pi) € and consider w^{p,T*). We will consider four cases. 

Case 1.1: Ai := {po < p^ A/r^}. Since p^ < p\ by ()3.9I) . the process ,t is a 

sub-martingale. Then 

E[U\p{t*),t*{p))1a,] = E[U\po,f\po))lA,] 

= 


1 . 




E 


poAp^Ap^ 


^plAp2 


Ul 


= E 

< E 

= IE (^?;^il{^i<^2}-bn^2l{^i>/,2}j 1 ai 
^ IE + G^2l{;,l>;,2}^ 1ai 

Case 1.2: A2 := {po = p^ A p"^}- We have that 

E[U\p{t*),t*{p))1a,] = E[U\po,f\po))lA,] 

= E[<U2] 

= E (^i?^il{^l<^2} +F^2l{;,l>;,2}) 1 a2 

< E (^i?^il{^l<^2} +Uj2l{;,l>;,2}) 1 a2 
- IE (-f^pll-{pl<p2} + G^2l{^l>^2}^ 1A; 
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Case 1.3: := {pQ > p} f\ p?} fl {p^ < p^}. Let f* := {T\,f^) € T^. We have that 


^[u\p{T*),T*{p))iA,] = w.[u\p{T*),r{p))iA,] 

= E[iK^i[u\p{r),r{p))]iA,] 

< E 


< E 


= E 






Case 1.4: A 4 := {pQ > A p"^} n {p^ > p'^}. 

E[u\p{t*),t*{p))1a,] = E[u\pi{p^),p^)lA,] 

— ^u2 1 a. 

= E 




By cases 1.1-1.4, we have (|3.10p holds. 

Part 2: We will show that 

w‘^{p*,t) < w‘^{p*,T*) 


(3.10) 


for any r € T^. We have that 

w‘^{p*,T*) = E[U^{p^,f‘^{p^))l{^i<^2} + U^{p^{p‘^),p^)l^^i^^2}] = E F^A{^i<f,2} + 

Take r = (tq, n) G and consider w‘^{p*,t). We will consider five cases. 

Case 2.1: Bi := {tq < p^ Ap"^}. On {t < p^}, > vf > AF^, and thus > Hf AF^ = . 

Moreover, since A > G^, 

p^ < inf{t > 0 : v'f = G^}. 

Hence, the process {v'^^^ 2 )t=o,... ,T is a sub-martingale. Then following the argument in the case 1.1, 
we can show that 

E [uHp*{r),T{p*))lB,] < E [(i"^'il{^i<^2j + Hl2l{,i^,2y) 1b, 

Case 2.2: B 2 ■= {tq = p^ A p^} D {p^ > p^} fl {p ^2 = We have that 
E[U^ip*{T),T{p*))lB,] = E[U\p^nip^))lB2] 


< E 
= E 

< E 
= E 


1^2 


^^1^2 


H. 2 IB 2 




where the third (in)equality follows from the definition of p ‘^2 in p3.7p . 
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Case 2.3: B3, := {tq = A /i^} \ ({//^ > fj,^} fl {p^2 = /^^})- We have that 
E [U^ip*{T),r{p*))lB,] = E [([/2(/,Ti(/ii))l|^i<^2| + 

Case 2.4: := {tq > A p^} fl {p^ < p^}. Following the argument in case 1.4, we can show 

that 

E [U^{p*{t),t{p*))Ib,] < E 1b, 

Case 2.5: B 5 := {tq > p^ A p^} D {p^ > p^}. Following the argument in case 1.3, we can show 
that 

E[uHp*{r),T{p*))lB,] <E[(F2a{^i<^2j + F22V>^2}) 1b, . 

From cases 2.1-2.5, we have (j3.10p holds. □ 


4. Comparison with the result in [T8] 

In this paper, regarding the existence of a Nash equilibrium in pure strategies, it leads to different 
results whether players act simultaneously or not at each stage. 

It can be expected that, in continuous time, as long as we have enough regularity for related 
processes, we would have the existence of an (e-) Nash equilibrium for the stopping game where 
one player acts first at each time. Unlike the case in discrete time, if we impose some (right) 
continuity assumption of t/* in {s,t), then it would not make too much difference whether players 
act simultaneously or not. Indeed, in |18] . by assuming the continuity of t/® in {s,t), we show 
the existence of an e-Nash equilibrium in pure strategies for any e > 0 for the stopping game in 
continuous time, where players act simultaneously at each time. 
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